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Applying the max-product (and belief-propagation) algorithms to loopy graphs is now quite popular 
for best assignment problems. This is largely due to their low computational complexity and impressive 
performance in practice. Still, there is no general understanding of the conditions required for convergence 

O . and/or the optimality of converged solutions. This paper presents an analysis of both attenuated max- 

product (AMP) decoding and weighted min-sum (WMS) decoding for LDPC codes which guarantees 

^ ' convergence to a fixed point when a weight parameter, /3, is sufficiently small. It also shows that, if the 

i> ; 

f~^ ^ fixed point satisfies some consistency conditions, then it must be both the linear-programming (LP) and 



m 



X 



maximum-likelihood (ML) solution. 



[~^ ■ For ((i„, dc) -regular LDPC codes, the weight must satisfy l3{dv — 1) < 1 whereas the results proposed 

by Koetter and Frey require instead that (3{dy ~ l){dc — 1) < 1. A counterexample which shows a fixed 



point might not be the ML solution if (3{dy — 1) > 1 is also given. Finally, connections are explored 
with recent work by Arora et al. on the threshold of LP decoding. 

Index Terms 

belief propagation, max product, min sum, LDPC codes, linear programming decoding 

L Introduction 

The introduction of turbo codes in 1993 started a revolution in coding and inference that continued 
with the rediscovery of low-density parity-check (LDPC) codes and culminated in optimized LDPC codes 
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that essentially achieve the capacity of practical channels HI, 121, [Sl, lH. During this time, Wiberg et 
al. advanced the analysis of iterative decoding by proving a number of results for the min-sum (a.k.a. 
max-product) decoding algorithm 151, ||6l, fT\. Richardson and Urbanke also introduced the technique of 
density evolution (DE) to compute noise thresholds of message-passing decoding algorithms for turbo 
and LDPC codes ISj]. 

For a particular noise realization, the optimality of iterative decoding solutions has also been considered 
by a number of authors. Weiss and Freeman have shown that the max-product (MP) assignment is locally 
optimal w.r.t. all single -loop and tree perturbations |9|. Unfortunately, this result is typically uninformative 
for LDPC codes with variables degrees larger than 2. Frey and Koetter have also shown that, with 
proper weights and adjustments, the attenuated max-product (AMP) decoding for LDPC codes returns 
the maximum-Ukelihood (ML) codeword if it converges to a codeword [7]. For general graphs, Wainwright 
et al. proposed the tree-reweighted max-product (TRMP) message-passing algorithm for computing the 
MAP assignment on the strictly positive Markov random field ifTOl . They have shown that, under some 
optimality conditions, the converged solution gives the MAP configuration for the graph. Their algorithm, 
though strictly different, has some similarity to the AMP algorithm in |7|. 

The linear programming (LP) decoding for LDPC codes, proposed by Feldman et al., solves a relaxed 
version of the ML decoding problem ifTTI . Since its introduction, a number of authors have looked for 
connections to the MP iterative decoding algorithm lfT2l . One interesting open question is, "What is 
the noise threshold of LP decoding?". The first threshold bound for LP decoding was proposed in (131 . 
Using expander graph arguments, they showed that LP decoding of a rate-i regular LDPC code can 
correct all error patterns with weight less than 0.000175 of the block length. Since this is a worst-case 
analysis, the large gap to the empirical observations is not too surprising. Daskalakis et al. lfT4l were able 
to improve the threshold to 0.002 using probabilistic arguments based on a construction of a LP dual 
feasible solution. In |[T5l . Koetter and Vontobel appUed girth-based arguments to the dual LP problem. 
For a (3, 6)-regular LDPC code, they proved that LP decoding can tolerate a crossover probability of 
p = 0.01 on the binary symmetric channel (BSC) and a noise level of o" = 0.5574 on the binary-input 
additive white Gaussian noise channel (BIAWGNC). 

Arora et al. showed recently that, for a (3, 6)-regular LDPC code, LP decoding can tolerate a crossover 
probability p = 0.05 on the BSC |[T6l . Instead of using a dual LP solution, they investigated the primal 
solution of the LP problem and proposed a local optimality condition for codewords. They proved that 
the local optimality implies both global optimality and LP optimality. So the probability that LP decoding 
succeeds is lower bounded by the probability that the correct codeword satisfies a set of local optimality 



conditions. Since their local optimality conditions are amenable to analysis on tree-like neighborhoods, 
they perform a DE analysis to obtain BSC noise thresholds for LP decoding. Using DE for memory less 
binary-input output symmetric (MBIOS) channels, Halabi et al. showed that LP decoding can achieve a 
noise threshold a = 0.735 on the BIAWGNC ifTTl . 

The results in this paper can be seen as an extension of the work by Frey and Koetter that provides 
new insight into results of |[T5l . |fT6l . We view both AMP and WMS I1I81I algorithms as computing the 
dynamic-programming (DP) solution to the optimal discounted-reward problem on a set of overlapping 
trees. This allows us to show that, for any received vector, the one-step update of the algorithm is a 
contraction on the space of message values when weight parameter is sufficiently small. From this, we 
deduce that the messages converge to a unique fixed point. We first show that, for (d^, (ic)-regular LDPC 
codes, if the resulting fixed point satisfies some consistency conditions, then it must also be the LP 
optimum solution and, hence, the ML solution. Then, the WMS algorithm on ((i„, tic) -regular LDPC 
codes with messages diverging to ±00 is considered. We show that, for the weight (3 = j^, if the 
WMS messages diverge to ±00 and satisfies the consistency conditions, the corresponding hard decisions 
also return the ML solution. 

The rest of the paper is organized as follows. Section Ull provides the background on factor graphs 
as well as the update rules of the AMP algorithm and the WMS algorithm. In Section |llll we first 
investigate the convergence property of both algorithms, and introduce the consistency conditions for 
both algorithms. Then, the optimality of the hard decisions corresponding to the consistent fixed point 
is discussed. In Section |IVl the optimality of the codeword returned by the WMS algorithm when the 
messages are not converged is analyzed. A conjecture about the connections between noise thresholds of 
the WMS decoding and noise thresholds of the LP decoding is proposed in the same section. Numerical 
results are described and discussed in Section |Vl Finally, conclusions and extensions are given in Section 

II. Background 

A. Factor Graphs 

An LDPC code can be defined by a bipartite graph Q = (V, f), where £ is the set of edges, and 
V = Vl U Vr consists of variable nodes (or bit nodes) Vl and check nodes (or constraint nodes) Vr. In 
this paper, ((it,,dc) -regular LDPC codes are considered. That is, each variable node in Vl has d^ edges 
attached to it, and each check node in Vr has dc edges attached to it. For a set S, let |5| denote the 
cardinality of S. The number of variable nodes denoted by n is \Vl\- Any binary vector x G {0, 1}" is 



a codeword, or a valid assignment, if and only if it satisfies all check nodes in Vr. We use C to denote 
the collection of all codewords. Let T/" be a computation tree of Q which has depth L and is rooted at 
node i. The set of vertices in the Ah level of 7^^, where ^ < L, is denoted by N{i, i). We also consider 
computation trees of Q rooted at a directed edge i ^ j or i -^ j with {i,j) G £■ For a graph Q, the size 
of the smallest cycle in G is denoted by girth(^). For a node i G V, we use N{i) to denote the set of 
neighbors of i. 

B. Discounted Dynamic Programming on a Tree 

Suppose that the computation tree T^^ has depth 2L < |girth(^). Then, each node in T^^ is associated 
with a different node in Q. Let X C Vl be the subset of variable nodes in T^^. A binary vector w € {0, 1}" 
is a valid assignment on T^^ if w satisfies all check nodes in 7^^. Let C'j-2l be the set of all valid 
assignments on T^^, and let Cq-2L[x) = {w € Cq-2L : Wi = x,'Wm = 0,V?n, ^ T^^} be a subset of 
C-J-.2L, where the assignment of the node ?n, G Vl \X is and the assignment of the root node is x. In 
the remainder of this paper, we often simplify C7-2L to Ct when i and L are evident from the context. 
Similarly, we also simpUfy Cj-2l(x) to C-j-{x). 

Let 7i(xj) = ^og{pY\x{yi\^i)) be the log-likelihood of receiving y^ G M given that Xi G {0,1} is 
transmitted at the ith bit. We consider the problem of finding the best assignment w* G {C7-(0) UC7-(1)} 
to a tree 7^^ defined by 

w* = argmax y^ Pmlmjwm), (1) 

t«G{Cr(0)UCr(l)}„g2: 

where /3m = /3^ if rri G N{i, 2£) and otherwise. Since Ct{0) and Ct{1) are disjoint, ([T) can be separated 
into two subproblems. For x G {0, 1}, we first define a vector 

w*{x) = argmax ^ Pm7m{wm), (2) 

and a function 

mix) = max y^dm^miwm), (3) 

■weCrix) ^ 

where /ij(x) is the optimal reward for assigning x to the root node of 7^^, and w*{x) is the cor- 
responding best assignment. Then, the solution of ([U is obtained by choosing w*{x*) where x* = 
argmax^gjo ij /Ui(3;). Note that ^j(-) is only a function of the assignment to the root node of T^^. 
Therefore, finding the best assignment of the tree 7^^ is equivalent to finding the best assignment of the 
root node of T^^. 



The RHS in ^ can be rewritten as 

L 
mix) = ji{x) + max V" V" P'^lmiWm)- (4) 

"- ' e=i meN{i,2e) 

This suggests that we can compute fii{x) recursively by using DP. In the {£ + l)th iteration, we compute 
the optimal discounted total reward fj,\^j (x) of assigning x to the directed edge i —>■ j by 

f^f+}\x) = j,{x) + p Y. ^'fU^) 

keN(i)\j 
= li{x) + p Y^ max Y /^m^fc(^™), (5) 

keN{i)\j • ^ ' m€N{k)\i 

where 

Sk,i{x) ^ |i(; € {0, 1}" : Wi = x, ^ ^™ = mod 2} (6) 

m&N{k) 

is the set of all valid assignments for variables in constraint k when x is assigned to the directed edge 
i ^ k. This follows from defining fJ-\J_i^{x) to be the optimal discounted total reward for assigning x to 
the directed edge i ^ k according to the rule 

^^ltki^) = max Y /^m^fc(^m)- (7) 

mGN {k)\t 



Finally, the reward function Q can be computed by 



^^i{x) = 7i(a;) + /3 ^ f^\Jj{x). 

j€N{i) 

To initialize the process, we choose fi\_^Ax) = ji{x) for all edges i ^ j and all x € {0, 1}. The update 
rule in ^ is the same as the AMP algorithm proposed in I?], where the optimal discounted total rewards 
fJ'i2^j{x) and filJ_Ax) are the messages passed on the directed edges i — )• j and i ^ j, respectively. 

By using the update rule Q, one can compute /ii(x) for all i G Vl in parallel. Suppose that the total 
number of iterations L is less than jgkth{Q). The vector x* with x* = argmax^gjQ ^^i fii{x) is the best 
assignments of the root of the trees {7^^ : i G V^}. Since 7^^ for all i G Vl are overlapped, any 
variable node i GVl appears in more than one tree. In both Q and lfT6l . it has been shown that if the 
best assignment of each i E Vl is consistent across all trees, then x* is the ML solution. To check the 
optimality of x*, one has to first find the best assignment w* of each computation tree, and then test 
whether the assignment w* of each tree is consistent with x* or not. In this paper, we discuss how the 
weight factor /? affects the decoder. We also propose other consistency conditions, which are easier to 
check, for regular LDPC codes. Finally, the analysis is extended to L > igirth(^). 



C. Attenuated Max-product Decoding Algorithm 

In Section III-BI the original AMP algorithm was introduced. In this section, we introduce a modified 
version of the AMP algorithm, which is mathematically equivalent to the original one for any finite 
number of iterations. 

Let 7j = 7i(0) — 7i(l) be the channel log-likelihood ratio (LLR) for the ith bit. It can be shown that 
w*{x) defined in dD also maximizes the following objective function 

IJ.i{x) = ^ Pm{l-Wmhrn- (8) 

To show the equivalence between the objective function of ([2]) and ([D, we subtract a constant XlmGX /5m7m(l) 
from the objective function of Q. Then, 

argmax ^ /3m7m(u^m) = argmax ^ l^mlmiWm) - ^ /3m7m(l) 

= argmax ^ /3m (7m(w^m) - 7m(l)) 

weCrix) „gj 

= argmax ^ /3m (1 - Wm) 7m- 
Therefore, the modified AMP update rule becomes 

f^ttjH^) = a - ^H + p E /^Sffc(^) 

k€N{i)\j 



= (l-x)7i+/3 Yl ™^^, , J2 f'J^ki'^rn), (9) 

keN{i)\j • ^ ' ■m£N{k)\i 

where the message ^J'\^j (x) now represents the weighted correlation between the LLRs and the best valid 
assignment with x assigned to the directed edge i — )• j. The algorithm starts by setting fj,-J^Ax) = {l—x)ji 
for all {i,j) e £ and all x e {0, 1}. 

Similar to the analysis in Section ITl-BI fii^j{x) can be considered as a DP value function, that assigns 
a real number to each bit-to-check directed edge i — )• j and each possible assignment x E {0, 1}. Based 
on the standard approach to DP, the update process can be seen as applying an operator A : M?'^' -^ M?'^' 
to messages. Let fi G M^l^l with fi = {/ij_j.j(x) : {i,j) E £,x G {0,1}} be an AMP message vector. 
From ^, the operator A is defined by i/ = A[/i] with 

Ui^j{x) = (1 - x)7i + /3 E f^i^kix). 

keN{i)\j 

= (l-x)7i+/3 y~] max V" ^lm^k{Wra)- (10) 

— wdSi (x) — 
k&N(i)\j ■ ' meN(k)\i 

The AMP algorithm proceeds iteratively by computing /^(^+^) = A[/i(^)]. 



D. Weighted Min-sum Decoding Algorithm 

Instead of passing the vector (^j^_|^ (0), ^j^WAV)) € M^ as the i ^f j message in the AMP algorithm, 
the WMS algorithm passes message fi\Xj — /Uj-_j.,(0) — yU^-j!^ (1), which is simply the difference between 
the best 0-root correlation and the best 1-root correlation. Similarly, the i ■^ j message is simplified to 
f^iJ-j — l^ii-j{^) ~ /^i4-i(l)- The update rules of the WMS algorithm are therefore given by 



kGN{{)\j 



''^''--+P E /^SJ.' (11) 

n 



m(^N{j)\i 






(12) 



It is easy to verify that the WMS algorithm is equivalent to the AMP algorithm. 

Similar to the AMP algorithm, for any WMS message vector /i € M'^' with /i = {^i^j : (i, j) € £}, 
the update rule of the WMS algorithm can be seen as an operator W : R'^I — > M'^', which is defined by 
i^ = W[/x] with 

i'i^j=-fi + (3 Y] W sgn(/xm_^fc) mill \nrn'^k\- (13) 

\ / m'^N(k)\i 

keN{i)\j \m&N{k)\i / ^ " 

The WMS algorithm is initialized by setting /U^-^ = 7j and proceeds iteratively by computing fj,^^~^^' = 
W[/xW]. 



E. LP Decoding 

Given the received vector y G W^, the ML decoder finds a codeword a;* G C such that the probability 
p{y\x*) is maximal among all x ^ C. Let 7 € M" be the vector of channel LLRs. Then, ML decoding 
can be defined as the following integer programming problem ifTTl . 

subject to X ^ C. 
For a fixed graph Q, solving (fT4l ) directly is computationally infeasible for large n because the number 
of codewords grows exponentially in n. In lillil . a suboptimal decoder, i.e., LP decoder, was proposed. 
With the same objective function as in (fT4l ). the LP decoder searches the optimal solution over a relaxed 
polytope which is obtained by intersecting all local codeword polytopes defined by each check node of 
the graph Q. 

Here, we briefly describe the LP decoder in ifTTI as follows. Given a check node j G Vr, let 

£j = {S C N{j) : \S\ is even} 



be the collection of all support sets of local codewords for j. Note that G £j and represents the all- 
zeros codeword. For each j € Vr, and S £ £j, Cj,5 is an indicator function of the local codeword being 
assigned to j. The LP decoder solves the following problem 

min ^ 7iXi 

subject to ^ Cj,S =1 V j G Vr 

See, 

^ 0,5 = a;i V(i,j) G £■ 

0,5 >0, Xi>0 Vi € Vl, Vi G Vr, \fS G £•,. 
If the solution vector x* is in {0, 1}", then the vector x* is an ML codeword. In the sequel, this LP 
problem is called Problem-P. 

To establish the dual problem of Problem-P, a Lagrange multiplier tjj is associated with each edge 
(i, j) G £" of the graph Q. The resulting dual problem is given by 

max Y^ Tj 

subject to /Jtjj > Tj Vj ^Vr, VS G £j 

y~] Tij < 7j Vi G Vl, 

jGN{i) 

which, as shown in |[T5]| . is equivalent to 

max > min > r,- ^ 

^^ see, ^-^ '^ 
ieVfi iG5 

subject to Yj Tjj- = 7i Vz G Vl. 

In the remainder of this paper, this dual problem is called Problem-D. 
Consider a ((i^,(ic) -regular LDPC code, and let 

£= Jw G {0, l}"^— ^w, = mod 2 I (15) 

be the set of locally valid codewords. For each check node j G Vr, we define a vector tj = {tjj : i G 
N{j)}. Then the objective function in Problem-D can be written as ^ ^^^ min^og/; {w,tj). 

F. Impossibility of a General ML Certificate for WMS Decoding 

In this section, two examples are provided for showing that WMS algorithm with some j3 > -j-^ is 
not guaranteed to return an ML codeword. 



Example 1. In this example, the ML optimaUty of the codeword returned by the WMS decoder with 
P = 0.8 is checked. We consider a (3, 4) -regular LDPC code over the BSC channel with cross-over 
probability p = 0.1. The parity check matrix for the (3, 4)-regular LDPC code is 

/ 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1 \ 
0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 

0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 

1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 
H= 1,0,0,1,0,0,0,0,0,1,1,0 

0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 

1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 
0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1 

\ 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1 y 

Since the codeword length is short {n = 12), we are able to implement the ML decoder defined in (IT4l ). 
For the WMS decoder, 200 iterations are performed in decoding each block. After testing 10^ blocks, 
there are 90905 codewords returned by the WMS decoder. Among these codewords returned by the WMS 
algorithm, only 90850 codewords are the ML codeword. Therefore, codewords returned by the WMS 
algorithm with /3 = 0.8 cannot be guaranteed to be ML optimal. 



For the general case, the following example gives some intuition. 



Example 2. Consider a (d^,dc) -regular LDPC code with codeword length n, where dc is an odd 
number and d^ > 3. Assume the all-zeros codeword is transmitted. Let the channel output LLR be 
7 = (-1,...,-1). Consider the WMS algorithm with (3 > j^. 

At the beginning, all messages from variable nodes to their neighboring check nodes are n\_^j = — 1 
for i = 1, . . . ,n and j € N{i). Consider the message passed from the jth check nodes to its neighbor 
variable nodes, fj-i^j, i € N{j). Since the incoming messages are all equal to —1, the update rule of the 
WMS algorithm at the check node gives 






yk&NU)\i 



mm 

k'eN{j)\i 



(0) 



for all (i,j) S £. In the first iteration, the outgoing message from the ith variable node to the jth check 
node is therefore 



f^. 



(1) 



li + P E ^^flk>-^ + idv-l) 

k£N{i)\j 



dv 



1 



1. 
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Moreover, one can show that /Uj-j^ — )• oo as £ — )• oo. Thus, the hard decision output is an all-zeros 
codeword. Unfortunately, given this 7, we know that the ML output must be a nonzero codeword with 
maximal Hamming weight. Therefore, WMS algorithm cannot provide an ML certificate for /3 > -j^. 
One might worry that this effect may be related to ties between ML codewords, but these can be avoided, 
without affecting the above result, by adding a very small amount of uniform random noise to the channel 
output LLRs. 

in. Convergence and Optimality Guarantees 

In this section, the optimality of codewords obtained by the AMP algorithms and the WMS algorithms 
for LDPC codes is considered. We will show that the AMP algorithm converges to a fixed point when 
the weight factor I3{dy — l){dc — 1) < 1. Further, if there is a codeword which satisfies the consistency 
conditions and uniquely maximizes the converged value functions, it can be shown that the codeword is the 
ML codeword. Similar to the analysis of the AMP decoding algorithm, we first discuss the convergence of 
the WMS algorithm. Compared to the convergence analysis of the AMP algorithm, a weaker condition for 
the convergence of the WMS algorithm, /3{dy — 1) < 1, is obtained. We also show that, if the converged 
messages satisfy the consistency conditions, which are similar to the conditions for the AMP algorithm, 
the optimality of the WMS codeword is guaranteed. 

A. Attenuated Max-product Decoding Algorithm 

Before showing that the AMP algorithm converges to a fixed point when j5 < ,^ ^utd -iv ^^ ^^^^ 
introduce the following tool lemma. 

Lemma 3. For any two vectors f,g(z M", the following inequality holds 



max|/i -^i! > 

i 

Proof: See Appendix lAl 



max fi — max gi 



2\e\ 



(16) 



Theorem 4. The operator A is an \\-\\^ contraction on W ' ' if 



/3< 



(4-l)(4-l)' 

Proof: Let n,u £ M^'^' be two vectors of AMP messages, and let /i' = A[/i] and u' = A[i/]. By the 
definition of A in (ITOl ) and the fact that for any two vectors / and g over M, | X]i(/i ~5i)l ^ Si \fi~9i\' 



11 



l/x' — i/'ll^can be upper bounded by 



u' — I/' = B max 
' "°° xe{o,i}, (i,j)es 



keN{i)\j k'eN{i)\j 



< 



P max y^ |/ij^fc(2;) -i^j^fe(2;)| 

:re{o,i}, {i,j)e£- f-^ 
fceAf(i)\j 



(17) 



From dV]), the last term of the RHS in (fTTl ) can be rewritten as 



max V] ^J'm-.kiwm) - max V] J^m'^fc(u'^ 

m£N{k)\i m'£N{k)\i 



(a) 

< max 

i«e5fc,i(a:) 



m&N{k)\i m€N{k)\i 



< max V" l^m^fc(Wm) - l^m^fc(Wm)|, 

m£N(k)\i 

where the inequality (a) follows by Lemma [3] Thus, the RHS in equation (ITtI ) is upper bounded by 



(18) 



/3 max V" max V" \lJ-m^k{Wm) - l^m^k{Wm)\ ■ 

xe{o,i}, (»J)6^ .4-^,, . n,es,A^) j-^ . 
Since max(/ + gf) < maxgi + max /, (fTSl ) is further upper bounded by 

/3 V" max Iflm^k {Wm) - J^m^k {Wm)\ 

m(^N {k)\t 

= /3{dv - 1) {dc - 1) max |/ij^j (x) - v-i^j {x)\ 

x&{0,l},{i,j)&£ 

where the inequality (a) follows from the fact that /3(dc — l)((ii, — 1) < 1. This proves the theorem. ■ 

Remark 5. Combining Theorem |4] with the contraction mapping theorem shows that, for an arbitrary 
(dt,, dc) -regular LDPC code and any < /3 < ,^ ^ihd -iv ^^^ AMP algorithm converges to a unique 
fixed point denoted by /x*. That is /x^^^ ^ //* as ^ — )• 00, and /x* = A[/i*]. We note that this idea is very 
similar to the existence proof for optimal stationary policies of discounted Markov decision processes. 

For each (i,j) G 8, let x* € {0,1} be the assignment which uniquely maximizes ix*^Ax), and let 
X* € {0, 1}" be the vector returned by the AMP algorithm. For regular LDPC codes, it suffices to show 
the ML optimality of a;* if the following conditions hold. 

Definition 6 (AMP-consistency). The assignment {x* . : {i,j) € £} is called AMP -consistent if x* G C, 



''id = ^i- 
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Lemma 7. Consider a {dy,dc) -regular LDPC code, and choose /? < ,^ -i\(d ^iv ^^^ each edge {i,j) G 
£, let fi*^Ax) be the fixed point, and let x* • uniquely maximize /i*_^,(x). Then for any binary vector 

Y^ fJ-UjiHj)< Yl i'^ - Hj) 7^ + Hdv - 1) {dc - I) Yl /^*-^i «i) ' 
{i,j)e£ {i,j)es {i,j)es 

with equality if and only if {x* ■ : {i,j) G £} is AMP-consistent, and Xij = x* , for all {i,j) G £■ 

Proof: See Appendix IB] ■ 

Remark 8. From Lemma U\ we know that when the assignment {x* } is AMP-consistent, then 

r^^/'^' ^''' - 1 - /? (4 - 1) (4 - 1) ' ^ ^ 

where x* = x*j for all i G Vl and {i,j) G £. 

Theorem 9. Given the LLR vector 7 G W\ let the assignment x* ■ uniquely maximize fx*^Ax) for all 
{i,j) € £. If {x*.j : {i,j) G £} is AMP-consistent, then x* = {x* : i G Vl} is the ML codeword. 

Proof: We prove that x* is the ML codeword by showing that x* uniquely maximizes the correlation 

J2ieV (^ ~ ^1)7* o^^'' ^11 codewords in C. 

Consider any codeword x G C such that x ^ x*, and {xij} G {0, l}'^' be the corresponding binary 
vector with Xij = Xi for all j G N{i). From ^, we know 

V (1-Xij)7i= V fi*^j (xij) - (3 V max V fi*^^kiwm)- (20) 

(j,i)e£- (i,j)gf (j,i)e£- -'^ '''^•mGAf{fc)V 

fceAf(i)\j 

By the fact that ^ is also in Sk,i{xij), we have 

max V ^;;,^fc (wrn) > V ^;;,^fc {xm^k) ■ 

■^ •'mGN{k)\i mGN{k)\i 



Therefore, the RHS in (1201 ) is upper bounded by 



mgAf(fc)\j 

= (1-/3(4-1) (4-1)) Y /"*-^i(^M)- (21) 

(i,i)e^ 

Since x^ . uniquely maximizes ^/^ xg^- /U*_^ . (x), the RHS in (|2TI ) is less than 

(1-/3 (4-1) (4-1)) j; /^:^,Ki)- 

(i,i)6£ 
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Thus, we have 

Y.{i-x,)^,<^{i-i3{d,-i){d,-i)) Y. f^UjKj) 

iGVi " (i,j)e£ 

where (a) follows from (fT9l ). This shows that x* uniquely maximizes the correlation XlieV (^ ~ x-i^ji 
over all a; G C and is therefore the ML codeword. ■ 

B. Weighted Min-sum Decoding Algorithm 

Before showing the optimality of the WMS algorithm, we first introduce a consistency condition for 
WMS decoding. 

it) 
Definition 10 (WMS-consistency). Let ij\_^j be the message passed from the ith bit to the jth check in 

the Ah iteration, and n\^j be the message passed from jth check to ith bit, defined in (fT2l ). The message 

vector z^^^) is called WMS-consistent if, for each bit i G Vl, it satisfies 1) sgn{pil\^-) = sg-Q.{pi\J^-,) for 

3,j' G N{i), 2) sgn{^lfl■) = sgn{iifX-) for j G N{i), and 3) sgn(7i + /3EiGAr(i) i^^^S-j) = sgn(/iSj,) 

for/ G A^(i). 

When the WMS messages satisfy the WMS-consistency conditions, the following theorem shows that 
the corresponding hard decisions return a codeword. 

Theorem 11. If the WMS messages in the ith iteration are WMS-consistent, then the hard decisions 



li^-^EnL + P Y. ^^^)\] 



for i = 1, . . . ,n give a codeword. 



Proof: We prove this result by contradiction. Assume that x is not a codeword. There exists at least 
one unsatisfied parity check node. Let j G Vr be the unsatisfied parity check node and N{j) be the 
neighbors of j. Since X^igTVfi) ^i = I mod 2, we have 

-1= n sgn(7^ + /3 E ^'^r 

idNij) \ j'eN{i) 

= n sgn(^£i)- 
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Consider the message passed from the jth check to the ith bit. From the WMS update rule, 



/^ 






ymeN{j)\i 



M) 




JJ sgn (/ui:' .„• I I X mm 



sgn //^4 . X mm 



M^ 



m€N(j)\i 






/^ 



m—^j 



This contradicts the condition 2) of WMS -consistency. ■ 

Next, we consider the optimality of the solution returned by the WMS decoder. Similar to the analysis 

of the AMP algorithm, we first discuss the convergence of the WMS messages. When the WMS messages 

converge to a fixed point, we show that the corresponding hard decisions give an optimal codeword if 

the fixed point is WMS -consistent. 

To show the convergence of the WMS algorithm, we first introduce the following lemma. 

Lemma 12. Consider two WMS message vectors /x, i/ € M' L Let i € Vl, k € Vji and {i, k) € £, and 
define 



d. 



i,k 



n 



\rn£N(k)\i 



sgn(/irn->fcj I mm |/im'^fc| 
m'^N(k)\i 



Then, 



TT sgn(i/m_^fcj mm \ym'~^k\ 

-•■-f-,, / ■m'eN{k)\i 



max |/i,n_!.^ i^m-^k\ ^ "j,fc- 

meN{k)\i 



Proof: See Appendix ICl ■ 

To show the convergence of the WMS messages, it will suffice to show that the WMS operator W is 
an IHloo contraction. The following theorem provides a precise statement. 

Theorem 13. For all LLR vectors and message vectors, the WMS operator W is an \\-\\^ contraction if 

/3(4-l)<l. 



Proof: Using Lemma [121 one can upper bound ||W[/i] — W[i/']||j^ in a straightforward manner to 
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get 

||W[Ai]-WM||^< max /3 ^ d,^k 
{t,j)(^£ keN{i)\j 

< /3 (dy — 1) max di k 

{i,k)ee ' 

< /3 (4 - 1) max lum^k - i^m^k\ 

(i, k)e£ 
m£N(k)\i 

= P{dy-l)\\ti-u\\^. 
This implies that W is a \\-\\^ contraction. ■ 

Remark 14. Combining this with the contraction mapping theorem shows that, for an arbitrary {d^^dc)- 
regular LDPC code and any < /3 < j-^r. the WMS algorithm converges to a unique fixed point, 

^j\_^j —^ fJ'i^j and /J-l^j — )• fJ-i^j, as the number of iterations goes to infinity. 

For any WMS -consistent fixed point, there are two ways to prove the optimality of the hard decision 
output. One way is by looking at Problem-P directly, which has been shown in our earlier work |fT9l| . We 
generalize the definition of minimal T-local deviation in lfT6l to T > \gVL\h{Q). By using the generalized 
minimal T-local deviation, it can be shown that, if the fixed point is WMS -consistent, the corresponding 
hard decision bits also return a locally optimal codeword. By the fact that local optimality implies global 
optimal and LP optimal, the hard decision is an LP and ML codeword. A summary of |fT9l is provided 
in Appendix IB 

The other method, which is introduced in the rest of this section, is by examining the optimality 
in Problem-D. We construct a dual witness according to the method introduced in |[T5l . The following 
lemma shows that the vector r* € M'^', which is constructed from the fixed-point messages ^i*_^j and 
/i*^ , is a dual feasible point of Problem-P. 



Lemma 15. Consider the WMS algorithm with (3 < j^ over a {d^, dc)-regular LDPC code. The vector 
T* e RI^I defined by 

r*. = i- {fiUj -Hdv- 1) l^t^j) (22) 



''^ d 



'V 



is a dual feasible point of Problem-P. 

Proof: Fix a variable node i eVi- The sum of the dual variables on the edges incident to i is given 



16 



by 



Yl <J = T Yl {f^Uj - P (dv - I) f^*^j) 



4 






^^^^j ~ Yl ^^ 



i-ir-k 



jeN(i) \ keN(i)\j 

= li- 

This proves the lemma. ■ 

Remark 16. Compared to the construction in ||T5l . Lemma [T5] is a simplified version by just considering 
one-step update of the WMS messages. In lITSl . min-sum messages over L iterations are considered. For a 
computation tree Tf^ of depth 2L rooted at check node j, those min-sum messages are used to generate 
an assignment t{j, L) to edges in T^^. Koetter and Vontobel showed that the dual feasible point r* can 
be obtained by averaging t{j, L) over all j G Vr. Since the number of leaf nodes in a computation tree 
increases doubly exponentially, a weight factor a is introduced to attenuate the influence of the leaves of 
the computation tree. In our analysis, by the fact that the WMS messages satisfy a fixed-point equation, 
we simplify the construction and consider only the assignments on the top level of computation tree. 
Next, we will show that the proposed dual-feasible point r* is also a dual-optimal point in Problem-D 
if it is constructed from a WMS-consistent fixed point. 

For a J € Vr, let t*, G M'^'^ denote the assignments on the edges incident to j, {r* : i G N{j)}, and 
let /^* = {fJ.*^j : i ^ N [j)} be the set of messages to j. Without loss of generality, we can sort the 
vertices in N{j) by {ii, 12,- ■ ■ , idj such that |/i*^^j| < l/^i^^jl < • • • < |m*^ ^,|- With this order, t* 
is rearranged into a vector t G M.'^", where tk = r*^ for A; = 1, 2, . . . , dc. Also, we define two vectors 
/x, /I G M'^" with fj,i^ = ^i*^^j and //^ = ^4^, for A; = 1, 2, ... , dc, respectively. Given a vector /, we 
use sgn(/) to denote a vector which is composed of the sign of each entry in /. Finally, we use 1 to 
represent an all-one vector, and the dimension is determined in the context of equations. 

The following lemma shows that an affine function of sgn(^*) minimizes the inner product {w,t) for 
all 10 G £ when the fixed point is WMS-consistent. Recall that C is defined in ( |T5l ). 

Lemma 17. Consider a {dy,dc)-regular LDPC code. For some j G Vr, if the WMS algorithm with 
/? < j^ converges to a WMS-consistent fixed point fi*_^j and fi*^- for all {i,j) G £, then 

argmin(«;,r*) = - (1 -sgn(/x*)) . (23) 
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Proof: Since messages are WMS -consistent, we know that the RHS in (1231) satisfies the jth check 
node from Theorem [n] From ((22)) . the LHS in (l23l) can be rewritten as 



fc=l 



1 



(a) 2. 



X] ^'^ (/^fc - /3 (d„ - 1) %f/j 

k=l 

J]] u-fc sgn pfc j (l^fcl -/3(d„-l) |/Ifc|j , 



(24) 



IfJ'kl 



fiJ - Hdv -i)Ui 



where the equality (a) holds by condition 2) of WMS -consistency. From the update rule of the WMS 
algorithm, one can show that 

1^2]' when k = 1, 

I /i ^ I , otherwise. 
Thus, the summation in (l24l) becomes 

—wisgn[jl-ij [Jjl^\ - I3{dy- 1) 1^2!) 

-, da 

+ -i-^WkSgn(jlf,j (l^Ifcl -/3(4-l)|^i|j . 

" fc=2 

Since < /3 (d^ — 1) < 1, one can show that 

|A| - Hdv- 1) |/^2| < 
for k > 2. Thus, the minimum is achieved by choosing 

-(l-sgnp^.jj for A; = 2, 3,..., 4, 
> Wm mod 2 for k = 1. 

By the fact that ^(1 — sgn(/x*)) satisfies the jth check node, thus wi = ^(1 — sgn(^^)) = ^(1 — 
sgn(;u|^_^ )). This completes the proof. ■ 

Remark 18. The proof of Lemma [TT] employs part of the observation in the proof of lITSl Lemma 3]. 
Given a check node, the absolute values of all but one outgoing WMS messages are the same. The only 
different absolute value of the outgoing message will be passed along the edge that the smallest absolute 
value of incoming message was passed on. With this observation, we know that the corresponding binary 
value wi will depend on the other binary values W2t ■ ■ , Wd^ . Since min-sum messages are not guaranteed 
to converge, Koetter and Vontobel computed the dual feasible point using computation trees of depth 



Wk 
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greater than one. In order to offset the influence of the exponential weighting of the messages from the 
leaf nodes, a large initial value assumption is required. With this large initial value assumption, they 
showed that the constructed dual feasible point is an optimal point in Problem-D. 

Let T G M'^I and gir) be the objective function in Problem-D. Let w^^' = ^(1 — sgn(/i*)) be the 
local assignment to check j. By Lemma [TS] and Lemma [TTl one can show that the optimal value of the 
objective function in Problem-D given r* is 

To find the optimal solution of Problem-D, one needs to search over all r in the dual-feasible set and 
find the maximum of /^(t). Let the optimal value of Problem-P and the optimal value of Problem-D be 
/* and g* , respectively. Since r* is in the feasible set of Problem-D, it is obvious that g* > g{T*). In 
the following theorem, we show that if the fixed point /x* = {/^|__>j : {i,j) ^ ^} is WMS -consistent, 
the proposed dual-feasible point r* actually achieves the maximum, that is, g* = g{T*). Also, the 
corresponding hard decisions return an optimal codeword, i.e., an ML codeword. 

Theorem 19. Consider the WMS algorithm with /3 < ^ \-^ . If the message vector /x*^^' converges to a 
WMS-consistent fixed point, fj,*, then the hard decision bits x* G {0, 1}" with 






is a codeword. Also, x* is LP optimal and, hence, ML optimal. 

Proof: Let r* be a dual feasible point constructed as proposed in Lemma [TSl Let w^^^ G >C be the 
binary vector that minimizes the inner product {w,t*) over all w ^ C Then, from Lemma [TTl we know 
w^^^ = i (1 — sgn (fJ-*)) for each j G Vr. Since the fixed point yu* is WMS-consistent, by Theorem [TTl 
it can be shown that w'^^^ = {x* : i e N (j)}, where x* = ^ (1 — sgn (7^ + /3 X],gAr(i) K^j)) i^ the hard 
decision of the ith bit. 

In the following proof, we will show that x* is LP optimal by contradiction. Assume that x* does not 
minimize Problem-P, then we have 

/• < E 7.-? *=' E f E <i 



j&Vii \i£N{j) / iGV, 



R 



19 

where (a) follows from Lemma \T5\ and (b) is a result of the WMS-consistency conditions. But, weak 
duality implies that /* > g*, and this gives a contradiction. Thus, x* minimizes the primal problem, and 
hence, is LP optimal. Moreover, since x* E C, it is also an ML codeword. ■ 

Remark 20. Consider the WMS algorithm on a (d^,, (ic)-regular LDPC code with /3 < j^- From 
Theorem [191 we are able to check the optimality of the WMS solution by testing the WMS-consistency 
conditions . If the messages satisfy the consistency conditions, then the hard decision bits return an ML 
codeword. 

IV. Weighted Min-sum Decoding with /? = j^ 

We first introduce some notation and definitions. We denote the WMS messages {/^^_\-7 • ("^'J) ^ ^} 
with /3 = j^ in the ith iteration by a vector fi^^' G W^'. The hard decisions computed by fj,^^°' are 
denoted by a binary vector x^^"^ € {0, 1}". For the WMS algorithm with /3 = -j^ and < (5 < 1, we 
use the vectors fig ' G MI^I and /i^ G RI^I to denote the messages in the ^th iteration and the fixed-point 
messages, respectively. The collection of hard decision bits computed using /i| is denoted by a vector 
x'^ G {0, 1}". Moreover, for any WMS message vector /x G M'^', the vector |/x[ G M^ consists of the 
absolute value of each element of /i. For any two WMS message vectors n,u £ M'^', we use /x = i/ to 
denote that sgn(;Uj_>j) = sgn(i/j_>.j) for all (i,j) G £. When comparing two vectors, we use the partial 
order /x ^ i/ to denote fii^j > Ui^j for all {i,j) G £, and /j, ^ v to denote Hi^j > ^i^j for all 
(i, j) G £. In the sequel, {/x} and {/x^} denote sequences of WMS message vectors {n^^' : ^ = 1, 2, . . . } 
and {fig : ^ = 1, 2, . . . }, respectively. We extend the definition of the WMS operator in (fT3l) to W^ for 
/3 = -J— I- The conditions for the operator W5 to preserve the partial order of the absolute value of the 
WMS messages are introduced in the following lemma. 

Lemma 21. Consider a {dv,dc)-regular LDPC code and a particular LLR vector 7 G M". Let /x, i^ G W^' 
be two WMS-consistent message vectors. If fi = u and [/x| ^ \h'\ >z ] 1, then \\Ns[fj]\ >z \Ws[i^]\ and 
\Ns[ti] = W^M ^ /x. 

Proof: See Appendix ID] ■ 

When /3 = j^, one may observe three kinds of trajectories of the WMS messages. They can 

converge to a fixed point, oscillate, or diverge to ±00. In this section, we are interested in the case when 

the sequence of WMS message vectors, {/x}, is divergent and WMS-consistent. We formalize this case 

by the following definition. 
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Definition 22. A sequence of WMS message vectors, {/x}, is divergent and consistent if 1) for all 
{i,j) G £, the absolute value of the WMS message, l/u^^j^ |, goes to infinity, and 2) there exists an integer 
L > such that /x^^) is WMS -consistent whenever i > L. 

Given two positive integers Li > Lq, to simplify notation, we denote / = {Lq,Lo + 1, . . . ,Li} by 
/ = [Lq, Li] when it is clear from context that / contains integers. A property of the sequence of WMS 
message vectors, {/x^}, is introduced in the following definition. 

Definition 23 (Block-wise monotone property). A sequence of WMS message vectors, {/x^}, is said to 
have block-wise monotone property in interval / = [Lq, Li] denoted by BMP(/), if for all £ G /, 1) /i^ 
is WMS-consistent, 2) ^tf = ^lf"\ 3) [^lf[ h ^^l, and 4) |/xf '^| t |/^f "^ 



In the following analysis, we show that, if there is an interval Iq = [Lq, Li] such that the sequence 
of WMS message vectors, {/x^}, satisfies BMP(/o), then {/x^} also satisfies BMP(/fc) for all intervals 

Ik = [Lq + k{Li — Lq), Li + k{Li — Lq)]. We first show that if {/x^} satisfies BMP(/o), then {/x^} also 
satisfies BMP(/i), where h = [Li, Li + (Li - Lq)]. 

Lemma 24. Let 7 G M" be the received LLRs, and consider the sequence of WMS message vectors {fJ,^} 
of a {dv,dc)-regular LDPC code. Suppose there exists an interval Iq = [Lq, Li] such that {/x^} satisfies 

BMP(Iq), then 

^(r+LO ^ ^(.'+Lo) ^25) 



and 



for all f = 0, 1, . . . , Li - Lq. 






>- 






(26) 



Proof: We prove this lemma by induction. The base case, i' = 0, is obtained since conditions 2) 
and 4) of BMP(/o) are satisfied. 

For the inductive step, suppose that /xf ^+^') = iif"'^^'^ and |/xf ^+^'^| >z [tif°^^'^[. Since iif°^^'^ 
satisfies conditions 1) and 3) of BMP(/o), from Lemma 1271 we have 



/^5 



(Li+r+1) 



>- 






and 



(Li+^'+l) s_ (Lo+r+1) s_ (Lo+n 
1^5 — t^S — t^S 
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Since both the base case and the inductive step are proved, we know that (1251) and (l26l) hold for < 

f < Li - Lo. ■ 

Corollary 25. Let 7 € M" be the received LLRs, and consider the sequence of WMS message vectors, 
{/i^}, of a {dy,dc)-regular LDPC code. Suppose there exists an interval Iq = [Lq, Li] such that {ns} 
satisfies BMP(Iq). Then {/15} also satisfies BMP(Ii), where Ii = [Li, 2Li — Lq]. 



Proof FromLemmalU we know |/xf +^'^| ^ \fif^^"^\ >l ^^1 for all £' = 0,1,..., (Li-Lq) 
:e /if +' 
implies that 



Since fJ-f^^"'^ satisfies condition 1) of BMP(/o), we know nf^^'^ is also WMS -consistent. Also, (l25 



where the second equality in sign is by the satisfaction of condition 2) of BMP(/o). Finally, by ( |26l ). we 
have 

/'or _ _r ^^ 



(2L1-L0) 



4'-" 



Therefore, we conclude that {/15} also satisfies BMP(/i) for Ii = [Li, 2Li — Lq]. ■ 

Now, we extend the property to intervals /^ for all A; > 0. 

Lemma 26. Consider the WMS algorithm with /3 = 3— ry on a [dy, dc)-regular LDPC code. Let 7 € ffi" 
be the received LLRs. Suppose there exists an interval Iq = [Lq, Li] such that the sequence of WMS 
message vectors, {ns}> satisfies BMP(Lq). Then, for all i > Lq, one finds that 

h'-S — 1^5 



and 






- 5 



Proof We first define L = Li — Lq and L^ = Lq + kL ioT k = 0, 1, 2, ... . Then, [Lq, 00) = 

{Lq, Lq + 1,. . .} can be written as 

00 

[Lo,oo) = [J 4, 

k=0 

where /^ = [L^, L^+i]. The lemma can be proved by showing that {/x^} satisfies BMP(/fc) for any A; > 0. 
We will prove this statement by induction. 

The base case is obtained from the assumption when setting A; = 0. Next, we consider the inductive 
step. Suppose that {/15} satisfies BMP(/fc). From Corollary l25l we know {/15} also satisfies BMP(/fc+i). 
Thus, we know that /x^ ' has BMP(/fc) property for any A; > 0. ■ 
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In the following analysis, we show that there exist a 6 > and an interval / = [Lo,ii] such that 
{/x^} satisfies BMP(/) when {/x} is divergent and consistent. We first show that, for any integer L > 0, 
the WMS message /x^^) for £ € [0, L] can be approximated by {/x^} with 6 close enough to 1. 

Lemma 27. Consider a (d^, dc)-regular LDPC code. Given the LLR vector 7 G M", let {/x} and {^ls] 
be two sequences of WMS message vectors with f3 = ^ _-^ and /3 = ^ _^ , respectively. For any e > 
and integer L > 0, there exists a 5 E [0, 1) such that H^^^) — /x^ ||oo < ^far all i < L. 

Proof: See Appendix |El ■ 

Given that {/x} is divergent and consistent, Lemma [27l implies the existence of 6 and Iq = [Lq,Li] 

such that {/X5} satisfies BMP(/o). The choices of 6 and Iq are also suggested in the proof of Lemma l27l 

The following lemma shows the existence of Iq and 6 by finding a valid pair of Iq and 6 such that the 

sequence of WMS message vectors {/x^} satisfies BMP(/o), and hence, satisfies BMP(/fc) for any k > 0. 

Lemma 28. Given the received LLRs, -y € M", suppose that {/x} is divergent and consistent. There exists 
an interval Iq = [Lq, Li] and a 5 € [0, 1) such that {/x^} satisfies BMP(Iq). By Lemma \26\ this implies 
further that there exists an Lq and 5 such that 



^'P 



t nl^^i (27) 





and 

/^f = m("°) (28) 

whenever i > Lq. 

Proof: We first introduce a valid choice of the pair of Iq and 6. Then, ( |27l ) and ( |28] ) are followed 
immediately by Lemma |26l 

Since {/x} is divergent and consistent, it satisfies conditions 1) and 2) of Definition |22l Therefore, we 
can find an Lo > 2 such that, for alH > Lq: /x^ is WMS -consistent; |/xW| >z 2||7||ool; and /x^ = /x^-^"). 
Similarly, we can also find an Li > Lq such that |/x(^)| ^ (||/^^^''^||oo + 2||7||oo)l whenever £ > Li. 



i||7||oo and 




2e _^ 1 


Li(Li + l)||7||oo ^ Li(Li + l) 


/x(^)-A^f 


<e = 7Tll7l|oo 
CO Z 



'^ > 1 - r ,r . .Ml, II = 1 - ^ ,, , ,^ (29) 

so that 

(30) 
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for all £ G [Lq, Li]. Note that ^^ and Li > Lq > 2 imply (5 > j^ . With these choices of Lq and Li, 
we have 



Mf' 



;^ 



lu(^°)|| 

l/^ lloo 



+ 2||7l 
3, 



i,(-^o)|| _| ii^ii 

IP" lloo I „ II nloo 



1 



y 



y 



||l|{-'-o)|| _|_ ^||-,|| 1 1 

llf* lloo I r) II / lloo ' -"- 

IH' lloo T^ t I X 



Mf"' 



Since [/i^^^| >: 2||7||ool for all £ G [Lq, Li], we know 



/^5 



>- 



At 



W 



1 V II II -1 ^ T OO -, 



for all £ G [Lq, Li]. Also by the fact that |/iW| >z 2||7||ool and /x(^) = fi^^°\ we know that 



Mf = /.(^^ 



A.(^°) = mI'"') 



(31) 



(32) 



(33) 



for all £ E Iq. Since /i^^") is WMS-consistent, dSS]) implies that /jbf^ is WMS-consistent for all £ £ Iq 
as well. By (|3TI)-(|33]) and the fact that fig is WMS-consistent for all £ G /q, we conclude that {/x^} 
satisfies BMP(/o). From Lemma |26l we obtain (H?]) and (HH) directly. ■ 



Theorem 29. Consider a (d^, dc)-regular LDPC code and a particular LLR vector 7 E M". If the WMS 
algorithm diverges (i.e., the messages tend to itcxDJ to consistent messages for j3 = ^ _-^ , f/ien f/iere 
/i' a (5 G [0, 1) 5Mc/j f/iaf /f also converges to consistent messages whose hard decisions give the same 
codeword as the WMS algorithm for /3 = ^ _^ . In this case, the codeword is the LP optimal and, hence, 
ML codeword. 



Proof: From Lemma |28l we have shown that there exist a Lq > and a < (5 < 1 such that 
fig = fi^^' = /x^^") for all £ > Lq. Since ^^-^ < j^, we know the messages will converge to a 
fixed point /x| and /x| = fi^^°\ Since /x^^") is WMS-consistent, the converged message vector /xj is 
also WMS-consistent. Hence, for all (z, j) G £ 



sgn I 7i + /3 Y^ fi\ 

jGN{i) 



t^j 



sgn {uli^j) 



sgn fi. 



XLo) 



Sgn 7i + 



jeN{i) ) 
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For any i G Vl, the hard decision Xg- with /3 = j^ is 




From Theorem |39j we know that a;^ is LP and ML optimal. Therefore, the hard decision vector x^'^"' is 
also an LP and ML optimal codeword. ■ 

Remark 30. In this paper, we considered the WMS algorithm as a DP problem with discount factor 
(3{dv — 1) < 1. When j3 = j^ and the sequence of WMS message vectors {/x} is divergent and 
consistent, the WMS update is equivalent to an Markov decision process (MDP) problem with discount 
factor 1. Theorem |29] essentially states that WMS decoding always has the natural analog of a Blackwell 
optimal poUcy if {/x} is divergent and consistent according to Definition [22l 

A. Connections with LP Thresholds 

In this subsection, we connect the LP threshold estimation with both the WMS algorithm and the DE 
type analysis in llT6l . lITTl . We have shown that when the WMS algorithm with /3 < j^ converges to a 
set of consistent messages, the WMS algorithm returns a codeword which is LP optimal. Similarly, when 
the WMS algorithm with /? = j^ satisfies conditions 1) and 2) of Definition |22l the WMS algorithm 
also returns a codeword which is LP optimal. If the following conjecture is true, we can conclude that the 
threshold of the WMS algorithm with /3 = j^ gives a lower bound for the threshold of LP decoding. 

Conjecture 31. Consider the WMS decoding of (d^, dc)-regular LDPC codes with girth Vt (logn) over 
a BSC with cross-over probability p and let p* be the bit-error rate threshold for the WMS decoding 
with (3 = 3— Ti" Then, the WMS decoding diverges to consistent messages with high probability for all 
p <p . 

Remark 32. DE gives automatically that almost all messages diverge to consistent values (i.e., a BER 
threshold). Conjecture [3T] is that p* is also a word-error rate (WER) threshold. Conjecture [3T] has been 
tested via simulation, and we are currently pursuing a rigorous proof. 
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Example 33. Consider a (3, 6)-regular LDPC code over a BSC. From a DE analysis of the WMS 
algorithm (i.e., not the DE for local optimality proposed in ||T6]| ) with /3 = 1/2, one finds that the WMS 
algorithm will decode correctly when p < 0.055. 



Remark 34. In the Example |33l the LP threshold lower bound of 0.055 matches the best possible bound 
using techniques from [16|. The main improvement over [161 is that our analysis (under the conjectures) 
holds pointwise for any received sequence. 

V. Numerical Results 

The word error rate (WER) for the WMS algorithms and the probability of not converging to a set of 
consistent messages are shown in Figure [1] The solid lines are the WER of the WMS algorithm, and the 
dashed lines are the probability of not WMS -consistent. The simulation is conducted over a (3, 6)-regular 
LDPC code ensemble with n = 10^. Two weight factors, /3 = 0.49 and /3 = 0.5, are considered, and 500 
iterations are performed in decoding one codeword. Both the BSC and BIAWGNC are tested. As shown 
in Figure [U when /3 = 0.49, the WMS algorithm may converge to a set of not WMS -consistent messages 
even though the codeword is successfully decoded. However, when /3 = 0.5, those two probabilities 
become nearly identical. 

To get the lower bound of the LP decoding threshold, a DE-type analysis is employed in |(T6l and ifTTl . 
The lower bound provided by the DE-type analysis depends on /3 though and is plotted in Figure |2] It 
is worth noting that according to our simulation result, the best lower bounds, in all cases, are obtained 
when /3 = j^, and that there is no threshold effect when /? < ^ ^_-^ . The threshold effect does not 
occur because the density of the correlation between the best skinny trees and the channel output in |16j, 
IfTTl converges to a fixed point instead of diverging to ±00. 

The comparisons of the WER performance between the WMS algorithm and the TRMP algorithm are 
shown in Figure |3] For any strictly positive pairwise Markov random field (MRF) with binary variables, 
it has been shown that the fixed point of the TRMP algorithm always specifies an optimal dual solution 
lITOl . |[20l . The TRMP message update rules in logarithm domain is 



k£N{i)\j 



,(^+1) _ „ TT .„. (,,i^) 



\meN(j)\t 
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Figure 1. The WER (solid lines) of the WMS algorithm for (3, 6)-regular LDPC code and the probability of converging to 
inconsistent messages (dashed lines). 



where p < lis the edge appearance probability. An uniform edge appearance probability p 



n{l+djd^)-l 



is employed in our simulation. One can notice that these update rules are similar to the WMS algorithms. 
Although, the factor graph for LDPC code is not strictly positive, the optimality of the TRMP hard 
decisions is observed in a numerical simulation of a (3, 4)-regular LDPC code with n = 12. Thus, 
we take the TRMP algorithm into consideration, and compare its WER performance with the WER 
performance of the WMS algorithms. 

In this comparison, a (3, 6)-regular LDPC codes over BSC is considered, and the codeword length for 
both algorithms is n = 10^. Three weight factors for the WMS algorithm are tested: /3 = 0.5, which 
is discussed in this paper; /3 = 0.8, which has been shown to have best performance by DE analysis 
lUH; and /3 = 1, which is equivalent to the conventional min-sum algorithm for LDPC codes. All WMS 
algorithms perform 100 iterations in decoding a codeword. In TRMP algorithm, two simulations with 100 
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Figure 2. The lower bound of the LP decoding threshold for (3, 6), (4, 8) and (5, 10)-regular LDPC codes over BIAWGNC 

and BSC. 

iterations and 1000 iterations, respectively, for decoding a codeword are conducted. As shown in Figure 

m the WER performance of the TRMP algorithm with 1000 iterations is close to the WMS algorithm 

with /3 = 1. However, if the TRMP algorithm only performs 100 iterations in decoding each codeword, it 

becomes close to the WMS algorithm with /? = 0.5. The performance loss of the TRMP algorithm with 

100 iterations is caused by the insufficient number of iterations. Since the TRMP algorithm is not close 

enough to the converged point, the corresponding hard decision bits are not reliable. Although TRMP 

algorithm over binary alphabet has been shown LP optimal when the algorithm converges, finding the 

noise threshold of the TRMP algorithm is still an open problem. 
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Figure 3. WER performance comparisons for a (3, 6)-regular LDPC code over tiie BSC. 

VI. Conclusions and Future Work 

For ((it,, dc) -regular LDPC codes, both the attenuated max-product (AMP) algorithm and the weighted 
min-sum (WMS) algorithm are studied. By slightly modifying the objective function of the original AMP 
problem in Q to an equivalent problem in ([D, we show that the AMP messages will converge to a fixed 
point when /? < ,^ „ivrf „i-) - Further, a set of sufficient conditions (AMP-consistency) for testing the 
optimality of the AMP solutions is proposed. With the modified AMP problem in ([8]), we show the LP 
and ML optimality of the AMP solution by a simple proof if /3 < ,^ -utd -U ^^'^ ^^ fixed point is 
AMP-consistent 



Similarly, when the weight factor /? < -j-^, we show that the WMS algorithm converges to a unique 
fixed point. We also introduce the sufficient conditions (WMS -consistency) for the hard decisions of the 
WMS algorithm to be a valid codeword. By employing the construction of a dual feasible point of the 
LP decoding (Problem-P) in ifTSl . we show that if /3 < -j^ and the WMS algorithm converges to a 



29 

consistent codeword, we can simplify the construction by using the converged messages. Also, we show 
that the dual feasible point obtained by the converged messages is an optimal dual feasible point, and 
the corresponding hard decisions are the LP optimum as well as the ML solution. Based on the analysis 
of the WMS algorithm with /? < -j^, the optimality of the WMS algorithm with /3 = -j^ is also 
discussed. When the WMS messages with j3 = -j^ satisfy the consistency conditions and diverge to 
zbcxo, we show that the hard decisions is ML optimum as well. This result can be seen as the natural 
completion of the work initiated by Koetter and Frey in [21. Also, our results have interesting connections 
with the results of jTEl because their best LP thresholds also occur when j3 = j^ according to DE 
analysis. For weight factors /? > j^, we provide a counterexample which shows that it is not always 
possible to provide ML certificates for WMS decoding. 

In regards to future work, the most interesting open question is whether connections between LP 
decoding and WMS decoding can be extended beyond (3 = j-^. In ifTSl . Chen et al. studied the optimal 
attenuation factor for the WMS algorithm. For example, the best /? for the (3, 6)-regular LDPC code on 
the BSC is /3 = 0.8, and the corresponding threshold is p = 0.083. DE also shows that any extension 
beyond (3 = ^^-^ will provide an improved lower bound on the LP threshold. Moreover, the construction 
of an optimal dual-feasible point for the LP decoding on an irregular LDPC code using WMS messages 
is still unclear to us. Let dv,i be the degree of the ith bit. With the construction proposed in this paper, 
we need /3j = /3 < minj'gv^{^— j— j-} for all i G Vl to ensure the convergence of WMS messages 
and the optimality of the corresponding dual-feasible point. However, there exists no threshold for the 
WMS algorithm with this choice of /3i. Therefore, a general weighting strategy and the corresponding 
construction of the optimal dual-feasible point for irregular LDPC codes is still an open problem. Since 
the irregular LDPC code has been proved to be capacity-approaching in HI, we expect that the irregular 
LDPC code with general weighting scheme can improve current estimate of the noise threshold for the 
LP decoding over a rate-i LDPC code. 

Appendix A 
Proof of Lemma [3] 

Proof: Let ^, m < n be the integers such that /^ = maxj /j and g^ = maxj gi. If fi>gm, it can 
be shown that fi — gi > for alH = 1, 2, . . . , n. Thus, it follows that 

max \fi - gi\>\fe- ge\ = fe - ge > h - 9m = \fe - 9m\ ■ 
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On the other hand, if fi<gm, we still can have the same inequality by 

max \fi - gi\ > \fm - gm\ = Qm - fm > gm - h = \fl " ^ml • 

Therefore, we obtain (IT6l l. ■ 

Appendix B 
Proof of Lemma [7] 

Proof: By the definition of the DP value function in (|9l), we have 

y] ^i*-^i (^^ij) = y] (l-a;i,j)7i + /3 y max V ^;,_^^, (uj„) , (34) 

(ij)e^ (ij)G£- (i,i)e^ '■ '■' mGAf(fe)V 

fcGAr(i)\j 
where Sk^i{xij) is defined in Q. Since x* . maximizes ii*^Ax), the inequality can be obtained by 

simply replacing Wm in (l34l ) with x^ ^. Thus, we have 

(i,j)G£ {i,j)&S {i,j)^£,k&N{i)\j, 

meN{k)\i 

= Y. {^-XiJ)J^+(3{d,-l){dc-l) Y. ^'i^jKj)- 

iij)e£ (i,j)ee 

To show the equality, by substituting x* ■ into ([34]), we have 

y f^Uj {x*j) = y (l - xlj) ji + /3 y max V fi*^^^, {w„,) . (35) 

(ij)e£ {J,i)G£- (ij)6£ ■ ^ '''^mGiV{fc)V 

kGN{i)\j 

Since {x* } is AMP-consistent, there exists a vector x* £ C such that x* = x* for all i £ Vl and 
j G A^(0- J^y the fact that x* G Sk,i{x*j), the last term in equation (1351) is equal to 

Therefore, we obtain the equality. 



(i,j)e£,k&Nii)\j, 
meN{k)\i 



Appendix C 
Proof of Lemma [T2] 

Proof: Since nmGAf(A:)V sgn(//m^fc) sgn(f„^fc) can be ±1, we must show that 



max \flm~^k - Vm-tk\ > 
m.GN(k)\i 



mm Ifim^kl - mm \Um^k\ 

m&N{k)\i meN{k)\i 



(36) 
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when all jim^k, ^m^k signs match on ?7i G N(k) \ i, and 



max \flm~^k -Vm-^k\ > 
m£N{k)\i 

when some signs differ. 
To show (l36l) . define the following indices 



min \l-im^k\+ mill \vm-^k\ 

m&N{k)\i meN{k)\i 



(37) 



mi = argmin \^irn^k\ , 



m£N{k)\i 

argmin 

m£N{k)\i 



m2 = argmin li^m^kl 



and 



Notice that 



m* = argmax l/u^^fc - i^m^fcl 

m£N{k)\i 



ll^'m'^k ^m*^k\ — max ||/im— >-fc| I'^m— )-A;|| 
meN{k)\i 

= max [l^'m— s-fcl ~ |/-*m— s-fcll • 
m&N(k)\i 

Consider the case when |^mi->fc| ^ I^»n,^A;I- Since l/ima^fel > l^mi-i-fcl > Wm2^k\^ it follows that 

l/^m*— >-fc ~ '^m*— s-fcl ^ max ll/im— >-fc| ~ l^m— s-fcl! 
meN{k)\i 

— N/^r?l2— )-fc| 1^^7712— )-fcl| 

^ ll/^mi-i-fcl — I'^ma^fcll • (38) 

When \iJ.m,^k\ < Wm2^k\, we know \iJ.m,~^k\ < Wm2-^k\ < Wm,^k\- It can be shown that 

max 11/Xm— >fc| ~ l^m— )-fe|l _ ll^mi— >fc| ~ |A*mi— >-fc|| 
meN{k)\i 

— \\^m2^k\ l/^rdi— >fc|| 

> WfJ'm.i^kl — Wm2^k\\ , (39) 

and combining the results in (1381 ) and ( [39l ) implies (l36l ). 

To show (|37] ). let A^ = {m € A^(/c) \ i : ^rn^k^m^k < 0} be the set of indices such that fim^k and 
ly-m^k have different signs. Notice that 
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max ifim^k ~ ^m,^k\ ^ max |/j,m_^fc — Z/'m_^fc| 

m£N{k)\i meM 

> max Wflm^kl + Wm^k\ 



> 



> 



min Ifim^kl + min Wm^kl 



min \fj.m~^k\+ min \i^m^k\ 

Tn&N{k)\i meN{k)\i 



This completes the proof. 



Appendix D 
Proof of Lemma [211 

Proof: Let /x' = V\/s[fj] and f ' = W5[i/]. One can compute the sign of the check-to-bit messages for 
each edge {i,j) G £ with sgn(^j^j) = nmgAr(j)v sgn(^„_^j) and sgn{i^i^j) = n„gAf(j)\i sgn(z^^^j). 
Using this and fi = u, it follows that 

sgnipi^j) = sgn{ui^j) (40) 

for all {i,j) e£. 

Since /i, i' both satisfy the consistency conditions, we know that sgn(/ij^j) = sgn(^j^j') and 
sgn(t'j^j) = sgn(t'j^j') for all j, j' G N(i). Thus, for each (i, j) G £", ^i^„- and vl^, can be expressed 



as 

/iUj = sgn (/ii^j) sgn ifii^j) 7i + -^— -^ J]] 

\ " keN{i)\j 

and 

sgn {i^i^j) sgn (z^i^j) 7j + ^ _^ ^ 
\ " fceAf(i)^ 



min l/Xm'^fcl (41) 

m'(^N{k)\i 



u'i^j = sgn{ui^j) I Sgn {ui^j)ji + - > min |z^m'^fc| | • (42) 

' '^ — ' — m'£N{k)\i 

Since li^l >~ "'''}'" 1, we have 

sgn(i/i<__j)7i + V" min |j^m'-^fe| 

fceAf(i)\j 

> sgn (i^i4-i) 7i + IItIIoo > 0. (43) 

Hence, 



H^j\ 



sgn (ui^j) ji + V" mill \um'^k\ 

keN{i)\j 
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mm Ifim'^kl 

. rn'eN{k)\i 



and similarly, 

I fJ-Uj I = sgn {^li^j ) 7i + -j——^ Yl 

" k€N{i)\j ■ 

By \fj,\ >z \iy\ and (l40l ). we have |/x'| ^ |i/'|. 

Moreover, from (1411 ). (l42l) and (1431 ). the signs of /i^_^ and f,-^ satisfy sgn(z/j'^ ) = sgn(^'j^j) and 
sgn(/L(^_j^ •) = sgn(/ij^j). By the consistency property of /x and u, we know sgn(i/j' ■) = sgn(t'j^j) = 
sgn(fj_^j) and sgn(;U^^) = sgn(/Xj^j) = sgn(/ii^j) for all {i,j) e <S. This concludes the proof. ■ 



Appendix E 
Proof ofLemma[ 

Proof: From (fTTI) . the absolute value of the difference nfj^, — fJ^fl^j in the ^th iteration can be 
written as 






dy 



k€Nii)\j 

By triangle inequality, (|44b is upper bounded by 

(£-1) ,(e-i) 






(£-1) 

d„ - 1 ^ "^'^^'^' 
fc'GAf(i)\i 



(44) 



dt, 



-1) ,,(^-1) 



^ ' 1 ^ l^i^fc' 



d« 



k'(LN{i)\j 



k&N{i)\j 

From Lemma [H] and ([12]), we know Ifif',^' - fJ's'i'^kl ^ ^^^m(^N{k)\i l/"ml>'fc - ^^m^fcl- ^^^°' ^y ^^^ 



-1) ,,(^-1) 



fact that \fJ-\^J\ = min^gjv(fc')\« l^m^fc'l' ^^ '^^'^ further upper bound (l44l) by 

,(^-1) ,,(^-1) 



Y ^ „3,^?^^^, l/^m-^fe ~M5,m^A: 



1-5 



1 — U ^r-^ 

+ -, > mm 

k'eN{i)\j 



r"m-¥k' 



Since l^-!^,; - t^A^^A < WfJ-'^^ ^^ - /^^ ^^lU and l/i-!^,^'*! < ^UtIIoo for all {i,j) G £, we have 



^,^j f^SA^jl ^11/^^^ "-/^5 



i^j 






< 



dr. 



- y y 

kGN{i)\j 



a-i) _ ^ii-i) 



1-5 v- 



OO (ly 



dy 



171 



fc'eAf(i)\i 



< 



;.(-^) - ;x?-) 



+ (1 



71 



(45) 



Since the RHS of (l45l) is a constant with respect to (i, j) G £, one gets the recursive upper bound 

+ (l-5)£||7||oo. (46) 



/x(^) - /^r 



< 



;.(^-)-;xr) 
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Note that |/Uj-_| — /^^ j_^J = 0. For a given £ < L, we can apply (l46l ) recursively, and have 



/^(^) - /^f 






S(l-.)^<^llTl 



OO) 



for all £ < L. Therefore, for any fixed e > 0, if we choose 



L(L + 1)||7|U' 
then liifl^ - fxfj^jl < ll/iW - /if lloo < e for all £ < L. 



5>1- ,,, ,!'„ „ , (47) 



Appendix F 
Extensions of the Work in |fT6 



In this appendix, we briefly recall the main idea and statement in our earlier work in [19], and provide 
detail proves of lemmas, which were omitted in |fT9l. We extend the lemmas and theorems in llT6l to 
the case when the depth of the computation tree exceeds igirth(t?). With these extended results, another 
proof of the conclusion drawn in Section IIII-BI is obtained. 

Since a computation tree with depth greater than ^girth(^) is considered in this section, we generalize 
the definition in Section ITl-B I as follows. Let 7^^ = {lUj', £') be a depth-2T computation tree and rooted 
at iQ gVl, where I and J' are the set of variable nodes and the set of check nodes in T^^, respectively, 
and T > igirth(^). Let i' and / denote a variable node and a check node in 7^^, respectively. We say 
that i' is associated with the bit i £ Vl in G (denoted i' ~ i) if i' is a copy of i. Similarly, / ~ j denotes 
that / G JT is a copy of j € Vr. Moreover, we define two projections rj : I ^ Vl and 6 : J ^ Vr by 
r]{i) = {i E Vl : i' ~ i} and d{j') = {j G Vr : j' ~ j}. 

At first, we generalize the definitions from [5| and |[T6l Definition 1] as follows. 

Definition 35. Consider a computation tree 1^'^ = {TVJ J, £') of depth 2T > igirth((/) and rooted 
at iQ. A bit assignment u G {0, l}'-^' on T^^ is a generalized valid deviation of depth T at io £ Vl 
or, in short, a generalized T-local deviation at iq, if ui^ = 1 and u satisfies all parity checks in 1^^ . 
Moreover, it is a generalized minimal T-local deviation if, for every check node j G 7^^, at most two 
neighbor bits are assigned the value 1. Note that a generalized minimal T-local deviation at iq can be 
seen as a subtree of 7^^ of depth 2T rooted at iq, where every variable node has full degree and every 
check node has degree 2. Such a tree is referred as a skinny tree. If -ccr = (roo, . . . ^vut) G [0, 1]^ is a 
weight vector and u is a generalized minimal T-local deviation at zq, then u^™) denotes the w-weighted 



vr,: I M^"^^ I = > rot > Um- 
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deviation 

(^) -cotUi ifi(^N {io, 2t) and < t < T, 

otherwise. 

For any nj-weighted deviation u(^) on 7/^^, let the projection of u^^^ onto the code bit i € Vl be 

T 
t=0 mGAf(io,2t):m~i 

Likewise, we let vr (ii^^^) represent the vector whose elements are ttj (w^™)) for i G Vl. The weights 
are chosen to be zut = /3* for some /3 G [0, 1]. 

To extend the results of fTB] to the computation trees of depth / — )■ oo, we utilize the following fact 
that, for each io ^ Vl, the WMS algorithm computes the best assignment, Xj„, for the root of T^^ , 
and there is a corresponding best assignment x for the tree T^^ ■ In the following lemma, a weighted 
correlation between x and a generalized minimal /-local deviation is introduced. Since x is the best 
assignment, it can be shown that the weighted correlation is positive when the number of iterations is 
large enough. 

Lemma 36. Given the LLR vector 7 € M" , let the assignment x for the computation tree T^^, computed 
by the WMS decoding with j3 < ^— ry. be unique (i.e., there are no ties) after / — )• 00 iterations. Let x 
be the corresponding assignment for T^^ . For any generalized minimal I-local deviation, u, rooted at 
iQ and any T <^ I, let the T-level weighted correlation be 

n T 

Ul{x,u)^Y.Y.P' E (-lf'"^-7^, (48) 

j=l t=0 meN{io,2t):m^i 

where N{iQ,£) is the set of vertices in the £th level of T^^ ■ Then, there exists a Tq < 00 such that 
Uj{x, u) > for all T >Tq and for all u. 

Proof: Since x is the optimal WMS assignment for the computation tree T^J after / iterations, there 
exists an e > such that, for all generalized minimal /-local deviations u, we have 
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nil 
< -e + XI Zl ^* Zl (im e n™) 7i 

i=l \t=0 meAf(io,2t):m~j / 

i=l yt=0 ymGAf(io,2t):m~j m€N(io,2t):mr~^i 

i=l \i=0 m&N{io,2t):m^i ) 

= -e + Vl{x) + Ul{x,u) + R{x,u), 
where 

R{x,u) = Y,{ Y. 1^' E (-if-^-JTi- 

4=1 \t=T+l mGAf(io,2t):m~i / 

and Xm © ^m is the sum of £,„ and n^ modulo 2. The \R{x, u)\ can be upper bounded by 

n I 

\R{x,u)\<Y. E ^* E ^™i^*i 

i=l t=T+l meN{io,2t):m^i 

(n I 

Y.Y. P' E ^™ 
i=l t=T+l meAf(io,2t):m~i 



= ll7lloc( Y. /5*d. (4 - 1)*~M 

\t=T+l / 

< WlWoo fd,{d, - if-^ I Y. /^*(^- - 1)* ) 



Since /3 < -j^zii, it follows that R{x, u) — )■ as T — ;• oo. Therefore, we can choose a Tq < oo so that 
U^{x, u) > e — R{x, u) > for all T > Tq. This completes the proof. ■ 

Remark 37. Let x and u be as defined in Lemma [36l and let T > Tq. Since U^{x, u) > 0, it follows that 
V^{x(Bu) = Vj^{x) + U^{x,u) > V^{x) for all u. This observation implies that, when (3 < j^ and 
the number of iterations is large, the binary assignments of the leaf nodes are asymptotically irrelevant 
to the assignment of Xi^. 

The following extends the key result |fT6l, Lemma 4] to our generalized minimal local deviations on 
the computation tree. 
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Lemma 38. Let V he the fundamental polytope of an LDPC, and z ^ V he a LP solution of a hit-regular 
code. Consider the set of depth- 1 computation trees rooted at all non-zero variahle nodes. For these trees, 
there exists a distribution over generalized minimal local deviations such that the expected value, when 
projected onto the original Tanner graph, is proportional to the LP solution z. 

Proof: This fact was first observed in lIlTl Remark 22]. See Appendix |G] for a proof based on 
extending the proof of [16, Lemma 4]. ■ 

The following theorem shows that if the WMS messages converge to a WMS -consistent fixed point, 
then the hard decision bits of the WMS algorithm give a codeword that is both LP optimal and ML. 

Theorem 39. For a given the LLR vector 7 G M" and a weight < /3 < j-ri- suppose the WMS 
algorithm converges to a WMS-consistent fixed point. If the hard decision bits x are unique (i.e., there 
are no ties), then they form a T-locally optimal codeword for some T < 00. Moreover, x is the LP 
optimal and, hence, ML codeword. 

Proof: From Theorem [TT] we know that ^ is a codeword. To prove that ^ is a T-locally optimal 
codeword, we have to show that for the projection 7r(u(^)) of any generalized minimal T-local deviation 
u^^\ the inequality 

(a; e cvr (ii(^)) ,7) > (a;, 7) 

holds, where c > is a scaling factor such that cTri{u^^>) < 1 for all i € l,2,...,n, and {x © 
C7r{u^^^))i = \xi — cTTi{u^^>)\ is as defined in llT6l . Without loss of generality, we assume that u^'^' is 
rooted at zq and consider the correlation of a; ® 7r(w(™)) and 7. This gives 



[x © CVr (■u(")) ,7) = ^ bi - CTTi (u^'^'^ 



li 



n T 

j=l t=0 meN{ia,2t):m^i 

= {x,'-f) + cUl{x,u), 

where U^{x,u) is as defined in ( |48l ). 

To show that Uf^{x,u) > 0, consider a tree 7^^ with large /. Since the WMS algorithm converges 
to a WMS-consistent message vector, the assignment for the subtree T^^ is the same as x for some 
T < 00. Here, Lemma |36] is required because the leaf assignment may not match a codeword. Also, u 



Un 
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can be obtained from the generalized minimal valid deviation u on 7^^ by truncating 

Um if w. G N{io, 2t) for some < t < T, 

otherwise. 

By Lemma [36l we can conclude that U^^{x,u) > 0. Therefore, i is a T-locally optimal codeword. 

According to |[T6l Theorem 4] or ifTTl Theorem 6], and by Lemma [38l the T-local optimality of x 
implies that x is the unique optimal LP solution given the LLR 7. Since x € {0, 1}" is an integer 
codeword, x is also an ML codeword. ■ 

Appendix G 
Another proof of Lemma [38] 

In this appendix, we extend the result of |[T6l Lemma 4] to the case when tree depth is greater 
than igirth(G). For a given non-zero LP solution z S [0, 1]", we first introduce the construction of the 
computation trees 7^^ {z) for all i £Vl with Zi > 0. Then, the distribution over skinny subtrees of T^^{z) 
is introduced. With the defined distribution, the symmetry property of the probabilities of a directed path 
and the corresponding reverse path in Q is discussed. Finally, we show that z can represented by a linear 
scaling of the expected value of bit nodes. 

For each i £ Vl and Zi 7^ 0, consider the depth-2/ computation tree 7^^ = (I U J,£'). Let i 
and j be the variable nodes and check nodes in T^^ , respectively. We first remove the variable nodes 
{i G I : z ,1^ = 0} and the edges incident to these variable nodes from T^^ ■ After the first removal, any 
nodes that are unreachable from i are removed as well. The remainder of the tree is denoted by T^^{z). 
Note that the distance from i to every leaf of T^\z) is also 21. 

To construct a probability distribution over all skinny subtrees in 7^^(z), we first define the transition 
probability between two distinct neighbors of a check node. For any check node j G Vr, the definition 
of the LP polytope implies that z can be rewritten as 

where TZj = {w £ {0, 1}" : J2ieN(j) Wi = mod 2}, a^ > and X^iueTe "^ ~ ^- "^^^ coefficient 
a-u, can be regarded as a probability distribution over TZj, and 

^i = ^ OwWi 
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is the expected value of the ith variable node of Q. For a j € Vr and an i ^ N{j) with Wi = 1, given that 
"j is reached from i", we define the probability of moving to m G N{j)\i (i.e. the transition probability 
from i to m) by 

p{m\i,j) = —pj{i,m), 

where Pj(i,m) = T.wen,,w.=ii^m'eNU)\i^^'^~^^^^"^- N°^^ ^^^^ 

Pj{i,m)= ^ Z^ y^ 

and, if Wm = 1, 



meNiJ)\i meN(j)\iw€nj ^m'eN{j)\i 



t(J„ 



Pj Vm, «) = 2^ ^^ = 2^ ^F^ = Pj [h m) . (49) 

•,£n, 2^m'eNiJ)\m'>^rn' ^^^^ 2^m'eN{j)\i'^rn' 



wi 



After having the transition probability, we then define a probability distribution over skinny subtrees of 
Tj^J {z). Let ^io(-^) 2/) be the set of all connected skinny subtrees of T^J {z). For a fixed r G Ai^iz^ 21), 
let Xr/ and jZr £ be the set of variable nodes and the set of check nodes in the ^th level of r, respectively. 
For each £ G {1,... ,/}, define B^{t) = {{i,j,rh) G 2^T,2(f-i) ^ Jr,2i-\ x 2t-,2£ : {hj-,i^) S ''"} as the 
set of all paths from the 2{i — l)th level of r to the 2^th level of r. The probability distribution over the 
skinny trees r G T^J {z) is defined by 

I 

Pio{'T)-j\ n p(^("i)h(^)'^o')) 

^=1 {i,j,»n)eB4r) 

=n n 7-/'e(i)(^«'^(^))- (50) 

^=l{«,i,m)GB£(r) ''(*) 

Let 62T ^ ^2T G Vl be a variable node at 2Tth level of T^^ {z). When a skinny subtree, r, of T^^{z) 
is randomly selected according to the distribution piair), the probability of having b2T in r is 

P^o{b2T)= Y. Pio (^') 1 (^2T G r') , (51) 

where l(-) is an indicator function, which is 1 if 62T is in r, and is otherwise. It is clear that there is 
a unique path in T^^ {z) from i^ to b2T- Let the path be b = (60, ^1, • • • , ^2t)> where 60 = ^o> ^21 G X 

for £ = 0, 1, . . . , T and 62^+1 G J for £ = 0, 1, ... T - 1. By substituting dSOjl into ([SB, we have 

T-i ^ 
Ko(^2t) = n Pe{b2w) {v{b2i),r]{b2i+2)) ■ (52) 



1=0 



^r]{b2i) 
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Let J2e+i = e{b2e+i) for £ = 0,1, . . . ,T - 1, and i2£ = vihe) for i = 1,2,..., T. The RHS of ^ 
becomes 

T-l 
P 



T-l ^ 
o(^2t) = n -—Phi+i fe£,i2£+2) • (53) 



Since (io)Ji)^2> • • • >j2T-i,^2r) also forms a directed path from io to ^2r in ^, and Zij^ = z^(p^^-^ > 
for all £ = 0, 1, . . . , T, there is a path c = (cq, ci, . . . , C2t) in T^J^ with cq = i2T. C2£ ~ i2T-2i for 
^ = 1, . . . , T and C2^+i ~ J2T-2e-i for ^ = 0, 1, . . . , T — 1. Note that the path c is associated with the 
reverse path of {io,ji,i2, ■ ■ ■ ,J2T-i: hr)- Similarly, by drawing a skinny subtree from Ai^^i^, 21), the 
probability of having C2t in the skinny tree is 



T-l 



K2t(c2t) = n Pe{c2i+i)ir]ic2e),vic2e+2)) 

T-l ^ 
= Y[ ~ PJ2T-2i-i («2T-2£,«2T-2£-2) • (54) 



From (l54l) . the probabilities pi^ (62t) and pi^^ (c2t) satisfy the symmetry property 

/ \ Y{(.=0 PJ2T-2t-i{'^2T-2i-2,'i2T-2t) 
Zi2TPi2T [C2T) = T-l 

11^' = 1 Zi2T~2l' 

(a) IlJ=0 PJ2e+i{i2£,i2£+2) 
T-iT-1 
lk' = l Zi2i' 

= Zi„Pia{b2T), (55) 

where the equality (a) is from ( |49b . and the equality (b) is from ( [53l) . 

For a variable node m £ Vl and a T < /, let 7Wm(r, 2T) be the subset of variable nodes associated 
with m and in the 2Tth level of a skinny tree r. The expected value of the size of M.^{t,2T) given 
r &T^/{z), denoted by Mm,i„i2T), is 

M„,i„ (2r) = Yl P^o {r) \M„,{t, 2T)\ . 

reAo(^,2/) 



Yl i Y. P'o (r) 1 (m € r) j 

JT:1'(z),2T) \reAn(z,2n / 



(a) 

meM,„{Ti'g'iz),2T) \TeA^a{z,2I) 

= Y Pioi^), (56) 

mGA4,„(7;f/{z),2T) 

where M.m{'T^J{z), 2T) is the set of variable nodes associated with m and in the 2Tth level of jfj {z), 
the equality (a) is from the fact that any m € M.mi'T: 2r) is also in the 2Tth level of 7^^(z), and the 
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equality (b) is from (ISTl ). In T^^ (z), the path from Iq to each m € A1m('7^^(z), 2T) is associated with 
a unique length-2T path from io to m in G, and the corresponding length-2r reverse path from m to io 
in ^ is also associated with a unique path from m to a variable node i G M.i^{T^ {z) ,2T) in T^{z). 
By (l55l) and (l56l ). we can have another symmetry property 



TeAo(2,2/) rh&M^{T{'^'{z),2T) 

= Z^ Y, PmiT)\Mi,{T,2T)\. (57) 

t£A,„{z,2I) 

With the above observations, we can start to prove Lemma [38l 

Proof: Let the probability of choosing i G Vl as the root of a skinny tree be p{i) = Zij \z\^. Then, 
for any i G Vl with Zi > 0, and any / > we can write 

E[X,] = YweY.piv) Y. Pv{r)\M,iT,2i)\ 
e=o vgVl TeA4z,2i) 

= E^^E^(^- E Pv{r)\Mi{T,2e)\]. (58) 

By dST]), the last term in the RHS of (l58]l is equal to Zi I]rGA{z,2/) Pi ('^) 1-^^ ('^i 2^)1- Thus, 






p,(r)|A^,(r,2^)| 

i{2,2/) / 

I 

When ^ is a {dv, dc)-regular bipartite graph, the number of variable nodes at the £th level of r is 

Y \MAr,2i)\=d4d,-iy-\ 
Thus, we have 

E[Xi] = Y^f,T^ Y Pi{r){d,{d,-lY-' 



1 + E ^^^-^ ^'^■" ~ ^^ 






and this concludes the proof of Lemma 
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